Beyond 2001: The Linguistic Spatial Odyssey

Why is it so hard to talk to a machine? If only we could communicate in a natural human language with robots, they would be so much more useful. Having machines that can reason spatially and receive and communicate such reasoning linguistically will extend their utility in many more scenarios that are dangerous, tedious, unhealthy, etc. Scene description, involving linguistic expressions of the spatial relationships between image objects, is a major goal of high-level computer vision. People have studied spatial relationships for several years. In a series of papers, we have introduced the use of histograms of forces to produce evidence for the description of relative position of objects in a digital image. There is a parameterized family of such histograms, for example, the histogram of constant forces (much like the earlier histogram of angles) and the histogram of gravitational forces that highlights areas that are close between the two objects. Utilizing the fuzzy directional membership information extracted from these histograms within fuzzy logic rule-based systems, we have produced high-level linguistic descriptions of natural scenes as viewed by an external observer. Additionally, we have begun to exploit the theoretical properties of the histograms to match images that may be the same scene viewed under different pose conditions. In fact, we can even recover estimates of the pose parameters. These linguistic descriptions have then been brought into an ego-centered viewpoint for application to robotics. We describe three initial activities here: production of linguistic scene description from a mobile robot standpoint, spatial language for human/robot communication, and understanding of a sketched route map for communicating navigation routes to robots. These efforts just scratch the surface of the potential applications and we end with future projections.